MULTIMEDIA QUERY SYSTEM USING NON-UNIFORM BIN QUANTIZATION 

OF HISTOGRAM 



BACKGROUND OF THE INVENTION 

5 

1. Field of the Invention 

The present invention relates to a multimedia query using a histogram, and more 
particularly, to a method for configuring a histogram which provides interoperability 
between histograms configured by non-uniform bin quantization using bits of a different 
10 number, and provides progressive bit processing for keeping a constant performance even 
when using the front parts of total bits of each bin value in the order of time series. 



2. Description of the Related Art 

Recently, as content-based multimedia query techniques are coming to the front, 

15 the study of multimedia features affecting a query performance is being made actively. 

Most frequently used search engines at present use global and local color information and 

texture information for image retrieval. Among them, the color information is known as an 

element mostly affecting the image retrieval. Thus, the development of more effective color 

features is being made, and also an attempt to develop color spaces more effective to the 

20 retrieval is being made. 

A color histogram is most widely used as color information. The color histogram is 

information representative of color distribution of multimedia data such as images. A bin 

number of the histogram is determined according to how the color space is quantized. 
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Although each bin value is represented by a decimal mostly, it also can be represented by N 
bits smaller than a fractional representation space for more improved performance and 
spatial efficiency. For example, it is generally known that spaces can be saved substantially 
without degradation of performance by representing 8 bits, i.e., decimals between 0 and 1, 

5 by 256 kinds of values. In a uniform quantization method, values between 0 and 1 are 
divided by a uniform width, quantized and represented, while, in a non-uniform 
quantization method, they are divided by a non-uniform width, quantized and represented. 

In case of using the non-uniform quantization method, it is possible to implement 
an improved performance as compared to the case of using the uniform quantization 

10 method or representing a decimal as it is. For instance, a section of an important bin value 
is divided more finely, while a section of a value having no division ability is divided less 
finely, for thereby enhancing the performance. For example, in case of a histogram, most 
bin values consists of a number smaller than 0.2, and thus it is meaningless to divide a 
number more than 0.2 finely. For this reason, the bin quantization using non-uniform 

1 5 quantization can be employed very usefully, but it may lead to the following problems. 

Problem of interoperability : Firstly, it is difficult to compare bin values each 
quantized by a different number. For example, a certain histogram quantizes and represents 
a bin value by 2"*=^ 16 in order to represent each bin value by 4 bits, while another histogram 
quantizes a bin value by 2^^4 in order to represent each bin value by 2 bits. Then, the 

20 comparison of these values is made impossible if there is no information about the method 
for quantization of each bin value. Therefore, it is necessary to ensure interoperability by 
which the comparison of bin values quantized by a different number by using a 
quantization method conforming to a particular protocol. 



Problem of progressive bit processing : when a histogram is transferred through a 
network, it can be transferred in the order of bins according to the coding scheme of the 
histogram, or in the order of bits of the bin. In case of transferring the histogram in the 
order of bins, every bits of the first bin are transferred and thereafter the next bin is 
5 transferred. In case of transferring the histogram in the order of bits, the first bit of the first 
bin is transferred and then the first bit of the second bin is transferred. In this way, after 
every first bits representing each bin value are transferred, every second bits representing 
each bin value are transferred. In such a coding scheme, if the transfer is interrupted before 
the histogram is completely transferred, the comparison of the histogram has to be enabled 
1 0 only by the transferred part of the histogram. 



SUMMARY OF THE INVENTION 

It is, therefore, an object of the present invention to provide a multimedia query 
using a histogram which enables comparison between histograms non-uniform bin 

15 quantized by a different number of bits, and also enables comparison between histograms 
by using the front parts of the histogram in the order of transfer when the histogram is 
transferred in the order of bits. 

To achieve the above object, a histogram in accordance with the present invention 
is provided, which comprises threshold values used in representing the number of n of bits 

20 for securing interoperability, capable of comparing the histograms represented by bits of a 

different number, including necessarily threshold values used in representing the number of 

N'(N'<N) of bits, when bin value is represented with the number of N of bits by 

quantization in order to query multimedia using histogram. 
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In addition, there is provided a multimedia query method in accordance with the 
present invention, v^hich comprises the steps of: correcting the other bin value as smaller 
number of N of bits and then comparing two values, if one of two histograms represents bin 
value with M, the other represents bin value v^th N(M>N), when performing the 
comparison between histograms represented with the number of bits which are differ from 
each other. 

Preferably, there is provided a multimedia query method in accordance with the 
present invention, wherein the correcting is performed by adding the bin values divided by 
the threshold values (THls) existing between two threshold values used in representing the 
number of N of bits to the rest threshold values except for threshold values (TH2s) used in 
representing the number of N of bits, among threshold values (THls) used in representing 
the number of M of bits. 

In addition, there is provided a histogram quantization method in accordance with 
the present invention, which comprises the steps of: dividing HMMD color space by a 
histogram, the histogram comprising threshold values used in representing the number of n 
of bits for securing interoperability, capable of comparing the histograms represented by the 
number of bits which are differ from each other, including necessarily threshold values used 
in representing the number of N'(N'<N) of bits, when bin value is represented with the 
number of N of bits by quantization in order to query multimedia using histogram, at this 
time, (a) producing indexes of two bin values by using a threshold value 2.5/310.0 for 
representing 1 bit bin value; (b) producing indexes of four bin value by using three 
threshold values 2.5/310.0, 9.1/310.0, and 30.0/310.0 for representing 2 bits bin values; (c) 
producing 16 indexes by deciding 0.0 of bin value as an index for representing 4 bits bin 



values, producing two indexes by bisecting a period of 0 and 0.6/310.0, producing an index 
in the period of 0.6/310.0 and 2.5/310.0, producing five indexes in the periods of 2,5/310.0 
and 19.0/310.0, producing six indexes in the periods of 19.0/310.0 and 85.0/310.0, and 
designating the value above 85.0/310.0 as an index; and (d) producing an index by deciding 
0.0 of bin value as an index for representing 6 bits bin values, producing four indexes by 
diving a period of 0 and 0.6/310.0 into fovir equal parts, producing four indexes by diving 
the period of 0.6/310.0 and 2.5/310.0 into four equal parts, producing ten indexes by 
dividing the periods of 2.5/310.0 and 19.0/310.0 into ten equal parts, producing thirty 
indexes by dividing the periods of 19.0/310.0 and 85.0/310.0 into thirty equal parts, 
producing fourteen indexes by dividing the periods of 85.0/310.0 and 121.0/310.0 into 
fourteen equal parts, and designating the value above 121.0/310.0 as an index. 

In addition, there is provided a histogram in accordance with the present invention, 
which comprises n-th bit represented by quantization of bin value, wherein the nth bit 
bisects the respective regions divided into (N-l)th bit in order to perform a progressive bit 
processing capable of retrieving by using only a bit, which is smaller than N, when bin 
value is represented with the number of N of bits by quantization in order to query 
multimedia using histogram. 

Preferably, there is provided a histogram encoding method in accordance with the 
present invention, which comprises the steps of: arranging firstly the first bit of every bin; 
and arranging the second bit, when encoding the histogram representing bin value capable 
of a progressive bit processing. 

Preferably, there is provided a multimedia query system in accordance with the 
present invention in which only a portion of the histogram is used according to the use of 



query and environment of a client, thereby capable of performing the optimum query. 



BRIEF DESCRIPTION OF THE DRAWINGS 

The above objects, features and advantages of the present invention v^ill become 
5 more apparent from the following detailed description when taken in conjunction with the 
accompanying drawings, in which: 

Fig. 1 is a view explaining an example of bin quantization which is interoperable; 
Fig. 2 is a view explaining a phase in which multimedia data represented as a 
histogram are transferred in the order of bit priority; 
10 Fig, 3 is a view explaining a partial query using a histogram of which transfer is 

interrupted; 

Fig. 4 is a view explainuig the relationship between a threshold and histogram data; 
Fig. 5 is a view illustrating a HMMD color space to which the present invention is 
adapted; and 

15 Fig. 6 is a view explaining a 184 level quantization method viewed in a MMD 

cross-section. 

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT 

The following detailed description of the preferred embodiments utilizes the color 

20 histogram only as an example for the purpose of illustrating the method for a multimedia 

query using the histogram in accordance with the present invention. The present invention 

is equally effective when applied to other types of histogram. 

A method for configuring a histogram non-uniform bin quantized and a multimedia 
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query using the histogram in accordance with the present invention having two objects of 
interoperabiUty and progressive bit processing will now be described. 

Firstly, achievement of interoperability in accordance with the present invention 
will be explained. 

5 With respect to histogram information of multimedia data, a color space can be 

quantized differently for various uses. Likewise, a bin value can be quantized differently for 
various methods for representing each bin value. For example, in a particular application, 4 
bits will be adequate for representing a bin value, while, in another application, at least 8 
bits will be adequate for the corresponding use. hi order to calculate similarity between 

10 those applications, it is necessary to ensure the result of the comparison between them 
using at least the same 4 bits. However, in the non-xmiform quantization, there are various 
quantization methods, so it is possible to compare them when the different quantization 
methods are employed. 

Accordingly, in the present invention, a method for quantizing a bin value to enable 

1 5 interoperability between them is suggested. 

Fig. 1 is a view explaining an example of bin quantization which is interoperable. 
In 101 of Fig. 1, a bin value is represented as 0 or 1 based on a particular threshold THl for 
1 bit representation. A histogram represented by such a method and 2 bit representation 
capable of direct comparison are described in 102 of Fig. 1. That is, a bin value is 

20 represented in four ways by using the threshold THl described in 101 of Fig. 1 and another 
two thresholds TH2 and TH3. In case of comparison with a histogram represented by 1 bit, 
since the threshold THl used in the 1 bit representation method is included as it is, the 2 bit 
representation can be corrected in the same way as the 1 bit representation by simply 



adding another parts. In this example, the total sum of section 2 102b, section 3 102c and 
section 4 102d of four sections 102a, 102b, 102c and 102d will be the same as the 
representation of 101. In this way, when a section is quantized more finely by using every 
thresholds used in dividing the histogram into smaller sections and another thresholds, even 

5 bin values represented by a different number of bits can be corrected and then compared by 
representation using bits of a smaller number. 

Next, the progressive bit processing in accordance with the present invention will 
now be described. Fig. 2 is a view explaining a phase in which multimedia data 
represented as a histogram are transferred in the order of bit priority. As shown in the 

10 drawing, a coding scheme can be considered in which a first bit of every bins (bin 1 - bin 
N) is transferred and then a second bit is transferred, not in the order of bins, in transferring 
a histogram. In this method, in the case that transfer is interrupted before the completion of 
the transfer as shown in Fig. 3, query can be executed only by transferred parts. At this time, 
in order to assure a certain degree of performance, when each bin is quantized by n bits and 

15 represented, it is necessary to represent optimum information which can be represented by 
each bit in the order of bits. For this purpose, the case of quantizing a bin value will be 
considered as follows. Firstly, assuming that a bin value is represented by 1 bit, one 
optimized threshold will be used to show the highest performance when the bin value is 
represented by two values, i.e., 0 or 1. In addition, in case of representing the bin value by 2 

20 bits, three optimized thresholds will be used to show the highest performance when the bin 
value is represented by four values. At this time, if only one bit of two bits can be used, the 
best performance can be expected when the optimized performance achieved at the above- 
described 1 bit representation is obtained. Therefore, in case of representing the bin value 
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by 2 bits, one of the three thresholds must be a threshold used in representing the bin value 
by 1 bit. 

In addition, the first bit indicates division of a value based on the above-described 
threshold of 1 bit as shown in Fig. 4. The second bit indicates division of the section of the 
5 value divided by the first bit. In this way, a n-th bit indicates division of each section 
divided by a (N-l>th bit. 

In other words, as shown in Fig. 4, if a bin value is represented by 1 bit in 401, it is 
divided based on a threshold Thl, If the bin value is represented by 2 bits in 402, each of 
sections divided by the threshold Thl is divided again by the previously used threshold Thl 

10 and another thresholds Th2 and Th3. If the bin value is represented by three bits in 403, 
each of sections divided by the previously used thresholds Thl, Th2 and Th3 and another 
thresholds Th4, Th5, Th6 and Th7. In this manner, when the bin value is quantized and 
represented by a small number of bits so that progressive bit processing can be achieved, a 
certain degree of query performance can be assured by using only transferred parts even 

15 though transfer is interrupted during the transfer of a histogram. Moreover, bin values of 
the histogram can be queried by using only parts of total bits according to the use of the 
query. At this time, in the present invention, even though parts of total bits are used, they 
represent every information of the entire color bins and thus a good performance can be 
expected. 

20 Fig. 5 describes a HMMD color space for explaining a progressive color histogram 

using the HMMD color space in accordance vdth the embodiment of the present invention. 
The HMMD color space is a color space of a double cone shape. The central axis thereof is 
represented as SUM ([MAX(RGB) + MIN(RGB)]/2), which corresponds to brightness. 



Fineness is increased in the order of center to peripheral sides of the cone, which is 
represented as DIFF(MAX(RGB)-MIN(RGB)). The angle of the cone indicates a color, 
which is generally represented as Hue. 

Fig. 6 illustrates an example of 184 level quantization of the HMMD color space 
explained in Fig. 5. 

As shown in Figure 6, first, a region is divided into 5 partial regions on the basis of 

DIFF and then subdivided on the basis of HUE and SUM again, thereby representing 184 

partial regions in all. These produce a color histogram constituted by 184 bins, respectively, 

as follows. First, indexes of two bin values by using a threshold value 2.5/310.0 for 

representing 1 bit bin value are produced. Thereafter, indexes of four bin values by using 

three threshold values 2.5/310.0, 9.1/310.0, and 30,0/310.0 for representing 2 bits bin 

values are produced. Also, 16 indexes by decidmg 0.0 of bin value as an index for 

representing 4 bits bin values, producing two indexes by bisecting a period of 0 and 

0.6/310.0, producing an index in the period of 0.6/310.0 and 2.5/310.0, producing five 

indexes in the periods of 2.5/310.0 and 19.0/310.0, producing six indexes in the periods of 

19.0/310.0 and 85.0/310.0, and designating the value above 85.0/310.0 as an index are 

produced. On the other hand, an index by deciding 0.0 of bin value as an index for 

representing 6 bits bin values, producing four indexes by diving a period of 0 and 0.6/310.0 

into four equal parts, producing four indexes by diving the period of 0.6/310.0 and 

2.5/310.0 into four equal parts, producing ten indexes by dividing the periods of 2.5/310.0 

and 19.0/310.0 into ten equal parts, producing thirty indexes by dividing the periods of 

19.0/310.0 and 85.0/310.0 into thirty equal parts, producing fourteen indexes by dividing 

the periods of 85.0/310.0 and 121.0/310.0 into fourteen equal parts, and designating the 
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value above 121.0/310.0 as an index is produced. 

In the present invention, in a multimedia query using a histogram, free comparative 
query between every servers on the internet, not one server, is enabled by providing 
interoperability by which comparative query between histograms bin quantized by different 

5 number of bits, when considering spatial efficiency by representing a bin value by bits of a 
decimal In addition, when the bin value is quantized and represented by a limited number 
of bits, a certain degree of query performance can be expected by executing query using 
only parts of the bits, for thereby enabling query at the time of transfer interrupt caused by a 
problem on a network and performing a query service conforming to the use of the query or 

1 0 the environment of a client. 
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